Skip to content

Update script to build and upload MiniBench artifacts #5017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Sep 5, 2024
Merged

Update script to build and upload MiniBench artifacts #5017

merged 11 commits into from
Sep 5, 2024

Conversation

kirklandsign
Copy link
Contributor

No description provided.

@kirklandsign
Add a new MiniBench apk for benchmarking
50492e7 
So far it supports generic model (not LLM). Later we can combine two

Build:
```
cd examples/demo-apps/android/MiniBench
mkdir app/libs
cp <executorch.aar> app/libs/executorch.aar
./gradlew :app:installDebug

When we have library changes ready, we can start trying it
adb shell am start -n org.pytorch.minibench/org.pytorch.minibench.BenchmarkActivity --es model_path /data/local/tmp/model.pte
adb shell run-as org.pytorch.minibench cat files/benchmark_results.txt
```
@kirklandsign
Android library update for MiniBench
e0dc99c 
Test:
```
cd examples/demo-apps/android/MiniBench
mkdir app/libs
cp <executorch.aar> app/libs/executorch.aar
./gradlew :app:installDebug

When we have library changes ready, we can start trying it
adb shell am start -n org.pytorch.minibench/org.pytorch.minibench.BenchmarkActivity --es model_path /data/local/tmp/model.pte
adb shell run-as org.pytorch.minibench cat files/benchmark_results.txt
```
@kirklandsign
Update script to build and upload MiniBench artifacts as well
152c5c5
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5017

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 570968f with merge base 61ddee5 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 30, 2024
@kirklandsign kirklandsign changed the base branch from main to aar-for-bench-2 August 31, 2024 00:09
@kirklandsign kirklandsign requested a review from guangy10 August 31, 2024 00:10
Base automatically changed from aar-for-bench-2 to main August 31, 2024 01:07
guangy10
guangy10 reviewed Sep 4, 2024
View reviewed changes
build/build_android_llm_demo.sh
}

build_android_llm_demo_app() {
build_android_demo_apps() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rename the script and steps in android-perf as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Let's do that later. It's really used in different places

guangy10
guangy10 approved these changes Sep 4, 2024
View reviewed changes
Copy link
Contributor

@guangy10 guangy10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me

@guangy10
Copy link
Contributor

guangy10 commented Sep 4, 2024

I think you will need #5098 in order to run on S22

@kirklandsign kirklandsign requested a review from huydhn September 4, 2024 23:57
@kirklandsign
Copy link
Contributor Author

We can use this one later with #5094 on generic models 😸

@kirklandsign kirklandsign merged commit 084659e into main Sep 5, 2024
45 of 52 checks passed
@kirklandsign kirklandsign deleted the update-script-minibench branch September 5, 2024 00:00
huydhn
huydhn approved these changes Sep 5, 2024
View reviewed changes
Copy link
Contributor

@huydhn huydhn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! After this PR and once I figure out how to get the results from the device, let's sync on what we need to change on the test spec to work with minibench

@guangy10
Copy link
Contributor

guangy10 commented Sep 5, 2024

@kirklandsign This PR is merged. I'm wondering how it's tested? I thought you will wait for #5098 then test the minibench in the android-perf using this PR.

@guangy10
Copy link
Contributor

guangy10 commented Sep 5, 2024

@kirklandsign This PR is merged. I'm wondering how it's tested? I thought you will wait for #5098 then test the minibench in the android-perf using this PR.

#5098 is merged. @kirklandsign can you please verify if the app works with QNN models?

@kirklandsign
Copy link
Contributor Author

@kirklandsign This PR is merged. I'm wondering how it's tested? I thought you will wait for #5098 then test the minibench in the android-perf using this PR.

#5098 is merged. @kirklandsign can you please verify if the app works with QNN models?

Oh let me pull in QNN deps and I will do a verification. One moment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@huydhn huydhn huydhn approved these changes

@guangy10 guangy10 guangy10 approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@kirklandsign @guangy10 @huydhn @facebook-github-bot